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IMPROVED METHOD AND APPARATUS FOR 
CREATING SOUNDS IN A VI RTUAL WORLD 

5 

BACKGROUND OF THE INVENTION 
This invention relates to virtual reality systems 
and, more particularly, to a method and apparatus for 
creating sounds in a virtual world. 

Users of computer systems are now able to create 
virtual realities which they may view and interact with. 
One type of virtual reality system is disclosed in U.S. 
patent application No. 535,253, filed June 7, 1990, entitled 
"Virtual Reality Network," the disclosure of which is 

15 incorporated herein by reference. One task which must be 

performed is the creation of the virtual worlds within which 
the users interact. The virtual world should simulate the 
real world as closely as possible. Thus, not only must the 
animated world be created, but the sounds which one would 

20 expect to exist in the virtual world must also be provided. 

SUMMARY OF THE INVENTION 
The present invention is directed to a method and 
apparatus for creating sounds in a virtual world. The 
25 system provides signal processing capabilities to convert 

monaural sounds to fully spacialized sound sources. A user 
of the system wearing a pair of stereo headphones perceives 
live, computer generated, or recorded sounds as coming from 
specific locations in space, just a listener does in the 
30 real world. 

BRIEF DESCRIPTION OP THE DRAWINGS 
Fig. 1 is a block diagram of a particular 
embodiment of an apparatus according to the present 
35 invention for creating sounds in a virtual world. 
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BRIEF DESCRIPTION OF THE APPENDICES 
Appendix 1 is a text description of an apparatus 
according to the present invention for creating sounds in a 
virtual world; 

5 Appendix 2 is another text description of an 

apparatus for creating sound in a virtual world; and 

Appendix 3 is a source code listing for a program 
used for creating sounds in a virtual world. 

10 DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 

Fig. 1 is block diagram of an apparatus for 
creating sounds in a virtual world. A more detailed 
description of the apparatus shown in Fig. 1 appears in 
Appendix 2. The following describes some of the 

15 capabilities of the system. 

AudioSphere contains several innovations, including: 
1: Acoustic touch feedback using spatialized acoustic cues 
for "Grab/Hit/Unhit" 
20 2: Simulated and exaggerated Doppler shift cues using MIDI 
PitchBend; 

3: Parallel processing architecture, where rendering and 
other computations happen in a separate processor, connected 
to the host by a low-bandwidth channel 
25 Another item: MIDI-based generation of real-time sound 

effects in VR. This item is a prerequisite for 2, and a 
subsystem in our implementation of 1 and 3, but MIDI sound 
in VR as such may be too general and obvious a method for 
any specific patent claim. 

30 

1: Touch Feedback 

Touch feedback is a valuable element of computer/human 
interface, particularly when using the hand to grab 
simulated or "virtual" objects, as with hand-measuring 
35 devices like the VPL DataGlove. The present invention uses 
sound rather than tactile feedback to indicate correct 
gesture for grabbing objects (Grab) , actual contact with a 
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grabbable object (Hit) , and release of a previously Hit 
object (Unhit or Release) . In our implementation, the sound 
is three-dimensionally rendered and appears to come from the 

• user's hand, but that need not be a requirement of the 

5 patent claim. Also, MIDI control of digitally sampled sound 

) is our synthesis method, but that should not be a 

prerequisite of the claim. 



In our invention, sound feedback indicates several things: 

10 Grab: whether the current hand gesture allows the object to 
be picked up (Grab gesture) . In the current implementation 
a grab gesture results in a continuous sound that continues 
until the hand intersects with a grabbable object. We use a 
sound of continual suction sound, **sssss", to indicate the 

15 hand's potential for picking up an object. This suggests a 
"vacuum suction" model of picking up objects, rather than 
closure of the fingers around the object, and helps the user 
make a correct assumption about the user interface. 
Hit: whether the hand has intersected with the object to be 

20 picked up (Hit) object can be grabbed now. In the Virtual 
Reality system, motion of the object now follows motion of 
the hand. The Hit sound can be continuous until the object 
is released, but in the case of the vacuum suction model, 
the sound is "sssspl" Another sound can continue while the 

25 object is being held, although in a system with other 
feedback (e.g., graphics) this is not necessary. 
Unhit: whether the Grab gesture has ended and the currently 
held object has been released. If the vacuum suction model, 
we use a sound of reverse suction, again followed by 

30. silence: "Psssss." 



35 
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2: Doppler Shift 

in the physical world. Doppler shift is the increase or decnsase of the pitch of a 
sound in accord wiih the speed of the object (rate of change of radial distance) 
relative to the listener. When a listener and object move toward eadtio\her, the 
5 pitch of a sound emanating from the objocl goes up when heard by the listener. 
When tiiey are moving away from eachother, the pitch goes down. The amount 
of pitch change is proportional to the fractional speed (rale of change of radial 
distance) of the objects relative to the speed of sound (at>out 600 miles per hour 
at common earth pressure and temperature). Thus the pHch of an object 
mo\dng toward the listener at 60 mph is raised bf about 10%. 

10 AudioSphere, in conjunction with Body Electric and its DM's, generates Doppler 
shifts by raising and lowering the pitch using MIDI PItchBend capability buitt in 
to many modem music synthe^zers. On synthesizers with polyphonic pitch 
bend capabilities, Bke the EMAX II synthesizer used fai ihe current AudioSphere, 
several different sound sources can be doppisr shifted at once. MIDI provides a 
low-bandwidth (typically 30 samples per second) method for the host computer 
and Body Electric to shift pilches of sounds emitted from objects in simulations, 

15 virtual reality, and other applications of Audio^here. 

MIDI is a hardware/software standard for generating and controlling sound in 
real>1ime on a low*bandMddth channel (31.25 Kbaud). MIDI PHchBand is a I4bit 
quanfi^ that takes a range of values from 0 to 1 6,383. The lowest downward 
bend value fs 0 and the highest pilch bend Is 16,383, with a middle value of 
8182 indicating no bend. 

20 

The Body Electric DMs allow tiie designer to specify the objects that have 
Doppler shifting, and to create attenuated or exaggerated doppler shifts as 
objects in the model move. The value for the PItchBend is detemtined by this 
formula: 

PItchBend « 8192-i'(ScaieFactor ' (Speed / SpeedOfSound) ) 
Speed is computed as the rate of change of radial distance between the object 

25 and the ear, u^ng the GiobalDlstance DM in Body Electric. Speed is positive 
when the distance Is increasing, negative when the object moves toward the 
listener. The sign of the ScaieFactor is negative so when two objects are 
moving toward eachother, the PitchBend value goes up. The ScaieFactor can 
be adjusted depending on the specific PitchBend response of the MIDI 
synthesizer, which typically ranges for 4-12% to •f-200%. The ScaieFactor or 
SpeedOfSound constants can be set to simulate very rapid motion. i.e. motion 

3 0 over great distances with a correspondingly dramatic pitch shift due to doppler 
when the object passes by. 

Exaggerated doppler ^Ift mtd exaggerated rolloff of sound loudness »v/f/7 
distance may be useful claims In an AudioSphere patent. Sound rolloffcan be 
proportional to the distance, the disumce squared, or any other exponent. The 
'cartoony exaggeraSons heighten the VR or other user's perception of space 
3 5 and motion in the application. 
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SrSSl?.!!!®*':'?.'®"^ cartesian coordinates lets the peripheral Drocafisor 
S muTh^I ?SffofL'r ? trigonometry without SSSfl the ffoTpJol^ssor 



While the above is a complete description of a 
preferred enbodiiDent of the present invention, various 
modifications «ay be eaployed. Consequently, the scope of 
the invention should not be limited except as described in 
the claims. 
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WHAT IS CIAIMED IS ! 

1. An apparatus for creating sounds in a virtual 
world comprising: 
5 coordinate means for providing cartesian 

coordinates of a sound-producing object located in a virtual 
world; 

transform means, coupled to the coordinate means, 
for transforming the cartesian coordinates to polar 
10 coordinates ; and 

sound generating means, coupled to the transform 
means, for generating a sound which is perceived as 
originating from the cartesian coordinates in the virtual 
world. 
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